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INTRODUCTION 


Giles  [10]  has  described  the  current  character  of  research 

in  -fuzzy  reasoning  as  -follows: 

"A  prominent  -feature  of  most  of  the  work  in  fuzzy 
reasoning  is  its  ad  hoc  nature.  ...If  fuzzy 
reasoning  were  simply  a  mathematical  theory  there 
would  be  no  harm  in  adopting  this  approach; 
...However,  fuzzy  reasoning  is  essentially  a 
practical  subject.  Its  function  is  to  assist  the 
deci si  on-maker  in  a  real  world  situation,  and  for 
this  purpose  the  practical  meaning  of  the  concepts 
involved  is  of  vital  importance"  (Giles  CIO],  pp.  263 

Fuzz-/  set  theory  would  benefit  from  becoming  a  behavioral 

science,  having  its  assumptions  validated,  and  having  its  results 

verified  by  empirical  findings  (k'ochen  [19]).  Unfortunately,  not 

too  many  researchers  in  this  field  have  adopted  this  philosophy. 

Except  fo*r  a  few  major  experimental  works  (Hersh  et .  al  .  [13,14]; 

Oden  [23,24,25,26];  Wallsten  et  al .  [31,32,33];  Rapoport  et  al . 

[281;  Zimmer  [36];  K'ochen  [19];  Zysno  [37]),  most  other  works 

center  around  axiomatic  treatment  of  the  subject.  As  noted  by 

Zeleny  C35],  it  happens  many  times  that  as  the  axiomatic 

precision  of  a  theory  increases,  it  becomes  less  significant  and 

relevant  in  its  impact  on  the  practice  of  human  decision  making 

and  judgment. 

This  paper  addresses  this  problem  in  the  area  of  linguistic 
approximation-  specifically,  we  focus  on  the  question  of 
selecting  an  appropriate  index  for  measuring  the  similarity 
between  fuzzy  subsets.  Several  methods  have  been  suggested  for 
the  process  of  linguistic  approximation  (Bonissone  C2D;  Eshragh 
and  Mamdani  C9];  Wenstop  C34]).  Each  of  them  suggested  a 
different  measure  of  similarity,  and  each  suffers  from  the  same 


ad  hoc  nature  as  most  of  the  other  work  in  this  -field.  There  is 
no  serious  attempt  to  validate  the  techniques  through  behavioral 
experiments.  Some  authors  have  mentioned  that  their  techniques 
work  very  well,  but  did  not  provide  the  appropriate  data  to 
support  their  claim.  For  example,  Bonissone  C21  in  his  pattern 
recognition  approach  to  linguistic  appro::  i  mat  i  on  writes  "This  new 
distance  reflects  very  well  the  semantic  distance  among  -fuzzy 
sets...  This  distance  has  been  applied  in  the  implementation  and 
has  provided  very  good  results,"  but  no  results  are  reported,  and 
it  is  not  clear  what  criterion  are  used  to  make  such  a  statement. 
Similarly,  no  serious  attempts  have  been  made  by  Wenstop  C34D  to 
validate  details  o-f  his  semantic  model.  Neither  do  Eshragh  and 
Mamdani  [91  behaviorally  validate  their  approach,  although  they 
claim  that  "...  the  results  obtained  -from  ’LAMS’  are  quite 
encouraging  and  also  considering  the  number  o-f  previous  attempts 
and  difficulties  involved,  one  can  say  that  ’LAMS’  has  proved 
workable,"  but  again  no  supporting  data  were  supplied.  More 
importantly,  no  attempt  has  been  made  to  compare  the  performances 
o-f  the  various  di-f-ferent  indexes  of  distance  that  could  be  used 
in  these  applications. 

Overall,  the  lack  of  behavioral  validation  for  any 
similarity  index  is  disturbing  because  of  (i)  the  crucial  role 
(translation)  that  this  index  plays  in  any  implementation  of 
fuzzy  reasoning  theory  and,  (ii)  the  relative  ease  by  which  any 
proposed  index  may  be  validated.  Regarding  the  second  point,  any 
successful  distance  measure  should  be  able  to  account  for  and 
predict  a  subject’s  similarity  judgment  between  fuzzy  concepts, 
based  on  his  separate  membership  functions  of  each  concept. 


The  notion  of  similarity  plays  a  -fundamental  role  in 
theories  o-f  knowledge  and  behavior,  and  was  dealt  with  broadly  in 
the  psychology  literature  (Gregson  C12D).  Overall,  the 
theoretical  analysis  of  similarity  relations  has  been  dominated 
by  the  geometric  model.  The  geometric  models  represent  objects 
as  points  in  some  coordinate  space  such  that  the  observed 
di ssi mi  1 ar i ty  between  objects  corresponds  to  the  metric  distance 
between  the  respective  points. 

The  similarity  indexes  used  in  the  linguistic  approximation 
techniques  adopted  this  approach.  Bonissone  C21  located  each 
concept  initially  in  four  dimensional  space,  where  the  dimensions 
are  :  Power,  Entropy,  First  moment,  and  Skewness  of  the 
membership  function,  and  defined  the  distance  between  two 
concepts  as  the  regular  weighted  Euclidean  distance  between  the 
points  representing  these  concepts.  Wenstop  C34D  located  the 
concepts  in  a  two  dimensional  space.  The  two  dimensions  are  : 
location  (center  of  gravity)  and  imprecision  (fuzzy  scalar 
cardinality)  of  the  membership  function.  The  distance  between 
any  two  concepts  in  this  space  is  the  regular  Euclidean  distance. 
The  same  geometrical  distance  philosophy  was  adopted  by  Eshragh 
and  Mamdani  C93  and  by  Kacprzyk  C163. 

Most  conclusions  regarding  the  appropriate  distance  metric 
have  been  based  on  studies  using  judgment  of  similarity  between 
stimuli  which  can  be  located  a-priori  along  (objectively) 
distinguishable  dimensions  (such  as  colors,  tones,  etc.).  The 
question  of  integral  vs.  separable  dimensions  is  crucial. 
Separable  dimensions  remain  subjectively  distinct  when  in 


combination.  By  contrast,  integral  dimensions  combine  into  a 
subjectively  nondecomposabl e  whole.  There  is  an  extensive 
literature  supporting  the  idea  that  the  Euclidean  metric  may  be 
appropriate  -for  describing  psychological  distance  relationships 
among  integral-dimensions  stimuli,  while  something  more  along  th 
lines  o-f  the  city-block  metric  is  appropriate  for  separable- 
dimensions  stimuli  (Attneave  C13). 

As  noted  by  Tversky  1303,  both  dimensional  and  metric 
assumptions  are  open  to  questions.  It  has  been  argued  that 
dimensional  representati ons  are  appropriate  for  certain  stimuli 
(those  with  a-priori  objective  dimensions),  but  for  others,  such 
as  faces,  countries,  and  personality,  a  list  of  qualitative 
features  is  appropriate.  Hence  the  assessment  of  si  mi  lari  tv  may 
be  better  described  as  a  comparison  of  features  rather  than  as  a 
computation  of  metric  distance  between  points.  Furthermore, 
various  studies  demonstrated  problems  with  the  metric  assumption 
Tversky  [303  showed  that  similarity  may  not  be  a  symmetric 
relation  (violating  the  symmetry  axiom  of  a  metric),  and  also 
suggested  that  all  stimuli  may  not  be  equally  similar  to 
themselves  (violating  the  minimality  axiom).  Therefore, 
similarity  may  be  better  modeled  by  a  function  which  is  not 
conceptually  a  geometric  distance  (e.g.  a  set-theoretic  function 
i nstead ) . 

The  plan  for  the  rest  of  this  paper  is  as  follows:  We  first 
review  the  various  distance  indexes  suggested  in  the  literature, 
in  the  general  case,  and  as  adapted  to  fuzzy  sets.  Second,  our 
experimental  design  will  be  presented.  Finally  we  will  discuss 


the  results  and  the  implications  o-f  the  results  to  the  process  of 


linguistic  approx i mati on . 

Geometric  Distance  Models. 

A  particular  class  o-f  distance  -functions  that  has  been 
investigated  by  psychologists  is  known  as  the  Minkowski  r-metric. 
This  metric  is  a  one-parameter  class  o-f  distance  -functions 
defined  as  follows: 


n 

d  ( x  ,  y )  =  C  £ 
r  - 


Ir]1/r. 


r  5  1 


( 1 ) 


where  x  and  y  are  two  points  in  an  n— di mensi oral  space  with 

\ 

components  (x^,y^)  i =1 , 2, . . . , n .  Let  us  consider  somE  special 
cases  that  are  of  particular  interest.  Clearly,  the  familiar 
Euclidean  metric  is  the  special  case  of  r=2.  The  other 

special  cases  of  interest  are  r=l  and  r=co.  The  case  o-f  r=l  is 
known  as  the  "city  block"  model.  As  r  approaches  w,  Eq.  (1) 
approaches  the  "dominance  metric"  in  which  the  distance  between 
stimuli  x  and  y  is  determined  by  the  difference  between 
coordinates  along  only  one  di mensi on-that  dimension  for  which 
the  value  lx^— y^ I  is  greatest.  That  is: 


d  <  x , y ) =max I x . -y .  I .  (1.1) 

W  11 

1 

Each  of  the  three  distance  functions,  r=l,2,  and  •»,  are 
used  in  psychological  theory  (Hull  C15D;  Restle  C29D;  Lashley 


[201 ) 


vnuT'nwin  ^ 


Generalizing  the  geometric  distance  models  to  -fuzzy  subsets. 

Let  E  be  a  set  and  let  A  and  B  be  two  -fuzzy  subsets  of  E. 
Define  the  following  family  of  distance  measures  between  A  and  B: 


r 

d  ( A,  B)  =  (E 

r  * 

i-' 


l,iA<::i 5  wB<Xi)l 


/n ) 


1/r 


r?l. 


(1.2) 


or ,  if  E=R 


dr(A,B)  =  </  i  W  A  <  ;<)  -JU.  <  x  )  Irdx)  1/r  r?l,  (1.3) 

"O 

and  d  ( A,  B)  =sup  I  ;i  .  )  -ji_  (>: )  I  .  (1.4) 

w  A  B 

The  cases  r=l  and  2  were  studied  by  Kaufman  [173.  Kacprzyk  [163 
proposed  the  distance  measure  (d0)‘,  and  d(  was  proposed  by 
Nowakowska  [223.  Our  empirical  evaluation  will  consider  d^,  d_,, 
(d„)^,  and  d  . 

go 

Hausdorff  metric. 

The  Hausdorff  metric  is  a  general i z at i on  of  the  distance 
between  two  points  in  a  metric  space  to  two  compact  nonempty 
subsets  of  the  space.  If  U  and  V  are  such  compact  nonempty  sets 
of  real  numbers,  then  the  Hausdorff  distance  is  defined  by: 

q(U,V)=max<  sup  inf  d2<u,v),  sup  inf  d2<u,v)>  <2) 

v«V  ueU  ^  ucU  veV 

where  d^  is  as  defined  in  Eq.  (1). 


In  the  case  of  real  intervals  A  and  B,  the  Hausdorff  metric 


■VUUIUVUIHUIUIUIURUIUIUIU 


nAnK  t>m 


is  described  by: 

q<A,B)=ma::C  la1-b1  I  ,  I  a-- -b2  I  >  (2.  1) 

where  A=Ca,.a^3  and  B=Cb , ,  b^3 . 

Generalizing  the  Hausdorff  metric  to  -fuzzy  subsets. 

Let  F(R)  be  the  set  of  all  fuzzy  subsets  of  the  real  line. 
There  is  no  unique  metric  in  F  <R>  which  extends  the  Hausdorff 
distance.  Ralescu  and  Ralescu  C273  proposed  the 
following  general i zat i ons: 

1 

q1(A,B)=|  q(Aa,Ba)da  (2.2) 

a=: 

q  ( A, B) =sup  q (A  , B  ) .  (2.3) 

Co  v .  a  or 

a  i’J 

where  A  is  the  a-level-set  of  the  fuzzy  set  A. 

We  propose  the  Hausdorff  distance  between  the  intervals  with  the 
highest  membership  grade: 


q*(A.B)=q(Ali0,B1-0). 


(2.4) 


If  A  and  B  are  real  intervals  then 

qx  <A,B)=qe9(A,B)=q>(AJB)=q(A,B)  . 


Goetschel  and  Voxman  C113  suggested  a  different 
generalization  of  the  Hausdorff  metric.  Let  A  and  B  be  two  fuzzy 
numbers  (for  the  exact  definition  of  fuzzy  numbers  in  this 
context,  which  is  slightly  different  from  the  usual  definition, 


see  Goetschel  and  Voxman  C113)  and  let  suppA=Ca. , b. 3  and 

H  H 


It 

If 


suppB=CaR,  b_]  and  let  a=min{a  ,a  '  and  b=max -Cb  ,b  >,  and  set 

D  D  H  D  n  D 


A  ={  (x ,  y )  I  a5:<  $b ,  0<yiu^  (x  )  } 


B  =  {  (x  ,  y )  I  a5>:  $b ,  0<y5tfB(x>J 


Then  their  distance  is: 


0<A,B)=qCA*,B*)  . 


(2.5) 


Dissemblance  index. 


Kaufman  and  Gupta  C183  start  with  distance  between 


intervals.  Let  A=Ca1,a0D  and  B=Cb.,b0j  be  two  real  intervals 

1  ’  jL  1 


1  ’  2' 


contained  in  CjS  and  define: 

1  ’ 


A  (A,B)  =  ( laj-bj 1+ la2-b2 I ) /2lfi2-fit ) .  (3.1) 


Generalizing  the  dissemblance  index  to  fuzzy  subsets. 


Now  let  A  and  B  be  two  fuzzy  numbers  in  R.  For  each  level 


a,  we  can  consider  A(Aa,Ba),  where  and  are  given  by  any 


convenient  values  which  surround  A  and  B  for  all  aetO.13. 

a  a  ’ 


Kaufman  and  Gupta  C181  now  define 


Aj(A,B)=J  a(A  ,B  )da. 

a-0 


(3.2) 


As  obvious  analogies  to  q  and  q  ,  we  add 

M  f 


a  (A,B)=supA(A  ,B  ) 
w  r  a’  a 

a 


(3.3) 


a4(A,B)=a(A1>0,B1>0).  (3.4) 


/.A-CcW..*, 
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Set  Theoretic  Approach. 

In  his  well-known  paper  "Features  of  similarity,"  Tversky 
1301  described  similarity  as  a  f eature-matchi ng  process.  The 
similarity  between  objects  is  expressed  as  a  linear  combination  of 
the  measure  of  their  common  and  distinct  features.  Let 
A  =  C  a, b,c, ...,]-  be  the  domain  of  objects  under  study.  Assume  that 
each  object  in  A  is  represented  by  a  set  of  features  or 
attributes,  and  let  A,  B,  C  denote  the  set  of  features  associated 
with  objects  a,b,c  r espect i vel y .  In  this  setting,  Tversky  C301 
derived  a;  i omati cal  1 y  the  following  family  of  similarity 
f uncti ons: 

s(a,b)=€'f  (AHB)-af  (A-B)-£f  <B-A> 
for  some  6, a, £50. 

This  model  does  not  define  a  single  similarity  scale,  but 
rather  a  family  of  scales  character i zed  by  different  values  of 
the  parameters  8,  a,  and  f> ,  and  by  the  function  f. 

If  a=£= 1  and  0=0  then  -s (a, b) =f <A-B) +f (B-A) ,  which  is  the 
di ssi on  1 ari ty  between  sets  proposed  by  Restle  C291. 

Another  matching  function  of  interest  is  the  ratio  model: 

f  (AflB) 

s(a,b)= -  oc,$}0, 

f  (AAB)  +otf  (A-B)  +0f  (B-A) 

where  similarity  is  normalized  so  that  s  lies  between  0  and  1. 
Assuming  that  f  is  feature-additive  (i.e.  f ( AUB ) =f (A) +f (B)  for 
AOB=0) ,  then  the  above  model  generalizes  several  set-theoretic 
models  of  similarity  proposed  in  the  literature.  If  oc=/?=l, 


s(a,b)  reduces  to  f  (AflB)  /f  (AUB)  (Gregson  C121).  If  a=5=<s  then 


s(a,b)=2f  (A/IB)/ <-f  (A)+f  (B)  )  (Eisler  and  Ekman  [73).  If  a  =  l  and 
£=0,  s  (a ,  b )  =f  (A/IB) /f  (A)  (Bush  and  Mosteller  E33).  Typically  the 
f  -function  is  taken  to  be  the  cardinality  function. 

Generalizing  the  set  theoretic  approach  to  fuzzy  subsets. 

Several  authors  have  proposed  similarity  indexes  for  fuzzy 
sets  which  can  be  viewed  as  general i zati ons  of  the  classical  set 
theoretic  similarity  functions  (Dubois  and  F'rade  C53).  These 
general i zati ons  rely  heavily  upon  the  definitions  of  cardinality 
and  difference  in  fuzzy  set  theory.  Definitions  of  the 
cardinality  of  fuzzy  subsets  have  been  proposed  by  several 
authors.  A  systematic  investigation  of  this  notion  was  performed 
by  Dubois  and  F'rade  [63.  For  generalizing  the  set— theoret i c 
approach  to  a  similarity  index  between  fuzzy  subsets,  the  scalar 
cardinality  measure  will  be  adopted  in  the  sequel.  The  scalar 
cardinality  (power)  of  a  fuzzy  subset  A  of  U  i s  defined  as: 

I A  1=1  ji^(u),  (De  Luca  and  Termini,  C43). 
uaUH 

When  Support (A)  is  not  finite,  we  define  the  power  of  A  to  be 

+  C0 

I A  I  =/  u^(x)dx. 

•M 

Define  the  following  operations  between  fuzzy  subsets: 

(a)  VxaX,  ji  -  _  <x  )  =  I  u  <>; ) -u  <x  )  I  . 

A#B  A  B 

A*B  is  the  fuzzy  subset  of  elements  that  belong  more  to  A 
than  to  B,  or  conversely. 

(b)  VxeX,  m  -  _  (x  )  =max  Cmi  n  (ji  (x  )  ,  l-ji<x  )  )  ,  mi  n  ( 1-y  (x  )  ,  y  (x))3. 

ft  B  ft  B 

AdB  is  the  fuzzy  subset  of  elements  that  appr ox i matel y  belong 


to  A  and  not  to  B,  or  conversely. 

The  -following  indexes  have  been  proposed  in  the  literature 
(Dubois  and  F'rade  C5D)  as  di  ssi  mi  1  ar  i  ty  measures  between  -fuzzy' 
subsets: 

(A,B)=l-JAflBI/IAUBI  (4.1) 

is  analogous  to  Gregson’s  C123  proposal  -for  classical  sets; 


( A, B) = I AoB I  (4.2) 

■il 

is  analogous  to  Restless  C29D  proposal  -for  classical  sets. 
A!  so 


£7i(A,B)=sup  (4.3) 

t  X 

and  -finally  a  di  sconsi  stency  index  ("degree  of  separation",  Enta 
C83  ) 


S4 (A, B) =l-sup  MAnB(x).  (4.4) 

x  e  X 


A  pattern  recognition  approach  (Bonissone  C23). 

In  this  approach,  the  efficiency  of  the  linguistic 
approximation  process  is  of  major  importance.  The  process  is 
composed  of  two  stages.  In  the  first  stage,  the  set  of  possible 
labels  is  narrowed  down  by  using  a  crude  measure  of  distance  that 
(hopefully)  performs  well  on  fuzzy  sets  that  are  far  apart  from 
each  other.  The  idea  is  to  represent  each  fuzzy  set  by  a  limited 
number  of  features  so  that  the  distance  computation  is 
simplified.  Bonissone  C23  chose  four  features:  (i)  the  power  of 
the  set  (area  under  the  curve),  (ii)  a  measure  of  the  fuzziness  of 


the  set  (non-probabi 1 i sti c  entropy)  defined  by  De  Luca  and 


Termini  C4D  as 

+<o 

Entropy (A) =/  S (u^ <>;  > ) dx 
■« 

where  S(y)=— yLn(y)  —  ( 1— y) Ln ( 1— y) , 

(iii)  The  first  moment  (center  of  gravity  of  the  membership 
function)  defined  by 

*  CO 

FMO  ( A)  =  (  J  x  u  A  ( x )  d>: )  /  Power  ( A >  , 

-f» 

and  finally  (iv)  Skewness  defined  as 

♦to 

Skew  ( A)  =/  (;iA(K)-FM0(A)  )  (>: )  d>: . 

"CO 

Bonissone  C23  defines  the  distance  between  two  fussy  sets  as  the 
weighted  euclidean  distance  between  the  vectors  (Power (A), 

Entropy (A),  FMO (A),  Skew (A))  and  (Power (B),  Entropy (B),  FMO(B), 
Skew(B)).  In  what  fallows  we  will  denote  this  distance  by  V^(A,B) 
(using  equal  weights).  After  narrowing  down  the  set  of  possible 
labels,  the  second  stage  starts,  in  which  a  modified 
Bhattacharyya  distance  is  computed.  This  distance  should 
discriminate  well  between  sets  that  are  close  to  each  other.  The 
Bhattacharyya  distance  is  defined  as: 

+c0  *  /  n  1 

R(A,B)=CW  (m  A(x)-ji  B<x>)  dxl  (5) 

•» 

where  the  membership  functions  have  been  normalized,  i.e.: 
W*a(x)=ma(x)  /  Power (A) 


and  similarly  for  u0. 


Wenstop  C34J  adopted  a  similar  approach.  He  represented 
each  -fuzzy  set  as  the  2-vector:  (Power  (A)  ,  FMO(A)>.  The  distance 
between  two  -fuzzy  sets  is  defined  to  be  the  regular  euclidean 
distance  between  the  two  corresponding  vectors.  We  will  denote 
this  distance  by  V7(A,B). 

Correlation  Index. 

Murthy,  Pal,  and  Majumder  C21D  defined  a  correlation-like 
index  that  reflects  the  similarity  in  behavior  of  two  fuzzy  sets. 
The  measure  is  actually  a  standardized  squared  euclidean  distance 
between  two  fuzzy  sets  as  defined  by  d0. 

Let 

Xft=J  <2ji  <x)-l)^dx 

'CO 

and  define 

CGRR ( A, B) =1- <4/Xa+X_) • <d„) .  (6) 

n  D  Z 

In  what  follows  we  will  use  the  index  p ( A, B) =1 -CQRR (A, B ) . 

METHOD 

Subjects:  15  native  speakers  of  English  were  recruited  by  placing 
notices  in  graduate  students’  mailboxes  in  the  business  school 
and. the  departments  of  anthropology,  economics,  history, 
psychology,  and  sociology  at  the  University  of  North  Carolina  at 
Chapel  Hill.  We  assumed  that  they  would  represent  a  population 
of  people  who  think  seriously  about  communicating  "degrees  of 
uncertainty,"  and  who  generally  do  so  with  non-numerical  phrases. 
The  general  nature  of  the  study  was  described  and  subjects  were 


promised  *25  -for  three  sessions  of  approximately  an  hour  and  a 
half  each. 

General  procedure:  Subjects  were  run  for  a  practice  session  and 
then  two  data  sessions.  The  experiment  was  controlled  by  an  IBM 
PC  with  the  stimuli  presented  on  a  color  monitor  and  responses 
made  using  a  joystick.  During  the  data  sessions,  subjects  woriec 
through  4  types  of  trials:  (1)  linguistic  probability  scaling 
trials,  (2)  Trials  in  which  subjects  integrated  two  probability 
terms  connected  by  AND,  (3)  Similar  trials  using  the  connector 
OR,  and  (4)  Similarity  judgment  trials.  This  paper  is  concerned 
only  with  tasis  <1>  and  (4).  For  more  details  about  the  other 
two  tasks  see  Wallsten  et.  al .  [333. 

(1)  Linguistic  probability  scaling  trials. 

The  objective  of  these  trials  is  to  establish  the  subject's 
membership  function  for  various  linguistic  probability  phrases. 
Recently,  Wallsten  et .  al .  C313  have  developed  a  method  for 

empirically  establishing  the  membership  functions  of  fuzzy 
concepts,  based  on  conjoint  measurement  and  utilizing  a  graded 
pair-comparison  technique.  Rapoport,  Wallsten,  and  Cox  C2S3 
further  established  that  the  methods  of  direct  magnitude 
estimation  and  graded  pai r-compar i son  yield  similar  membership 
functions.  In  this  study  we  have  adopted  the  direct  magnitude 
estimation  technique.  On  such  a  trial  a  red  and  white  radially 
divided  spinner  appeared  on  the  screen,  as  shown  in  Fig.  1. 


************************* 


Insert  Fig.  1  about  here 

************************* 


Instructions  -for  this  task  said  in  part: 

Imagine  that  you  cannot  see  the  spinner,  but  you 
have  to  predict  whenever  it  will  land  on  white  on  the 
next  random  spin.  A  friend  of  yours  can  see  the 
spinner,  although  not  too  well  because  it  is  rotating  at 
a  moderate  rate.  Your  friend  is  going  to  give  you  his 
or  her  best  opinion  about  the  chances  of  the  spinner 
landing  on  white.  However,  this  person  does  not  tell 
you  a  probability...  Rather,  the  person  may  use  any  of  a 
large  number  of  nonnumerical  probability  phrases...  We 
are  interested  in  your  interpretation  of  the  probability- 
phrases  as  they  apply  to  the  spinner  context.  Assume 
that  your  friend  tells  you  that  it  is  doubtful  that  the 
spinner  will  land  on  white.  This  gives  you  some  basis 
for  judging  the  probability  of  that  event.  Now, 
consider  the  spinner  on  the  screen.  How  close  is  that 
spinner’s  probability  of  landing  on  the  white  to  the 
judgment  you  had  formed  upon  hearing  that  it  is  doubtful 
that  the  spinner  will  land  on  white? 


The  subject  then  moved  the  cursor  on  a  line  to  indicate  how 
close  the  particular  displayed  spinner  came  to  the  opinion  that 
he  or  she  had  formed  on  the  basis  of  the  phrase  doubtful .  The 
cursor  could  be  moved  from  not  at  all  cl ose  (low  membership)  to 
absolutely  close  (high  membership). 

Six  phrases  were  employed,  three  representing  lower 
probabi 1 i ties  and  three  representing  higher  probabilities: 
doubtful .  slioht  chance,  i mprobabl e.  lively,  oood  chance,  and 
fairly  certain.  In  the  direct  estimation  task,  each  phrase  was 
presented  with  11  spinner  probabi 1 i t i es:  0.02,  0.12,  0.21,  0.31 
0.40,  0.50,  0.60,  0.69,  0.79,  0.88,  and  0.98. 

Subjects  judged  each  combination  of  phrase  and  probability 
numbers  twice,  once  in  each  session. 
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(2)  Similarity  judgment  trials. 


Instructions  for  this  task  said  in  part: 

...  Two  non-numer i cal  probability  phrases  will  be 
printed  on  the  screen  on  each  trial.  We  are  interested 
in  how  similar,  or  synonymous,  you  consider  the  two 
phrases  to  be  with  respect  to  describing  the  probability 
o-f  a  spinner  landing  on  white. 

The  subject  then  moved  the  cursor  on  a  line  to  indicate  how 

similar  the  phrases  are.  The  cursor  could  be  moved  -from  not  at 

all  s i mi  1 ar  to  absolutely  similar.  Each  subject  judged  the 

similarity  between  all  possible  pairs  (15)  (excluding  a  phrase  and 

itsel-f)  twice  in  each  session. 

Membership  -function  evaluation:  Based  on  previous  research 
(Wallsten  et .  al .  C311;  Rapoport  et .  al .  C2B3)  we  have  concluded 

that  a  cubic  polynomial  can  accurately  represent  the  membership 
functions  for  the  si::  phrases.  Note  that  a  cubic  polynomial 


i 


\ 

1 


i 

> 


resembles  the  ’S’  and  ’II’  functions  that  have  been  proposed 
the  literature  in  this  context  (Eshragh  and  Mamdani  C91>. 
cubic  polynomial  was  fit  to  the  22  paints  representing  each 
phrase  within  a  subject,  using  the  least  squares  technique. 


A 


i  n 


Each 


function  was  then  normalized  to  attain  the  value  1  on  the 


interval  CO, 13.  In  defining  the  membership  functions,  any  value 
less  than  zero  was  redefined  to  equal  0,  and  similarly  any  value 
greater  than  1  was  redefined  to  equal  1.  These  adjustments  were 
general  1  y  quite  minor.  Examples  o-f  the  membership  functions  for 
the  six  phrases  for  one  subject  are  shown  in  Fig.  2.  All 
membership  functions  for  all  subjects  were  either  nondecreasing, 
nonincreasing,  or  single-peaked. 
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Insert  Fig.  2  about  here 
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RESULTS  AND  DISCUSSION 

For  each  subject  and  each  pair  o-f  words,  all  nineteen 
distance  measures  were  calculated.  (At  times  it  was  necessary  to 
discretize  one  axis,  using  a  100— point  grid,  in  order  to 
calculate  a  distance  measure.)  To  evaluate  the  per-formance  o-f  a 
particular  distance  measure,  we  compare  its  computed  values  tc 
the  "true"  distance  ratings  as  given  directly  by  the  subject  in 
the  similarity  judgment  trials.  This  evaluation  is  done  on  two 
levels:  First  we  can  as!  i -f  the  distance  measure  correctly 

categorizes  a  “similar"  pair  of  words  by  returning  a  ‘'small" 
distance,  and  if  it  correctly  categorizes  a  "dissimilar"  pair  o-f 
words  by  returning  a  "large"  distance.  This  crude  evaluation  is 
in  practice,  independent  of  the  subject-specific  "true"  distance 
rating,  because  the  subjects  generally  agree  that  the  pairs 
p  (doubtful  .  i mprobabl e ) .  p„=  (doubtful .  si  1 qht  chance) . 

P-.=  (improbable,  si  1  oht  chance)  .  p^=(fairly  certain,  oood  chance) 
p^= (fairly  certain.  likely).  p  =  ( 1 l 1  el v.  good  chance)  are  each 

O 

composed  of  two  "similar"  words;  likewise  the  subjects  generally 
agree  that  the  pairs  g  = ( doubt  f  u 1 .  fairly  certain).  q„= ( doubt  f  u I 

1  di 

hance) .  a  = (doubtful  .  likely),  a  “(improbable,  fairl 


certain) 


(improbable,  aood  chance) 


(slight  chance,  fairly  certain) 


chance ) 


(si l oht  chance,  likely)  are  each  composed  of  two 


dissimilar"  words.  For  this  task  of  dichotomous  categorization. 


essentially  all  the  distance  measures  were  successful  across  all 
subjects  (see  Figure  3,  for  example).  This  is  testimony  to  the 
intuitive  base  upon  which  each  distance  definition  rests:  They 
are  designed  to  indicate  gross  differences  between  membership 
functions,  if  and  only  if  such  differences  actually  exist.  The 
practical  i mp 1 1  cat i on  is  simply  that  if  linguistic  approximation 
or  concept-clustering  is  to  be  carried  out  in  two  stages,  then 
any  of  these  distance  measures  may  be  used  for  the  first  stage. 

*»****»tt**t*»*»t****»* **» 

Insert  Figure  3  about  here 
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The  second  level  of  our  evaluation  asks  whether  the  distance 
measure  reflects  the  correct  degree  of  similarity  within 
"similar"  pairs  of  words,  and  whether  the  distance  measure 
reflects  the  correct  degree  of  di ssi mi  1 ar i ty  within  "dissimilar" 
pairs  of  words.  In  answering  this  more  subtle  question, 
intersubject  variability  must  be  acknowledged:  Each  subject  w. 1 1 

have  his  own  membership  functions  for  the  words  in  pair  p  . 

These  two  membership  functions  are  "similar"  in  the  gross  sense, 
but  the  similarity  between  them  is  different  than  the  similarity 
between  his  membership  functions  for  the  words  in  pair  p^.  The 
degree  of  similarity  within  each  pair  is  given,  for  that  subject, 
by  his  "true"  distance  rating.  If  the  distance  measure  works 
well  in  the  context  of  fuzzy  sets,  it  should  yield  distances  for 
pairs  p^  and  p^  that  "agree"  with  the  corresponding  "true" 
distance  ratings  given  by  the  subject.  To  quantify  the  amount  of 


18 


•/  v.v. /.  /. 


•.■.s’'-'.'- 


|T» 


agreement  between  a  particular  distance  measure  and  the  “true" 
distance  we  compute  the  correlation  between  these  two  quantities, 
over  all  pairs  (pi  :  l$i$&>,  -for  a  given  subject  (see  Figure  3). 

Thus  our  criterion  -for  agreement  is  linear  association.  Now,  the 
same  considerations  apply  to  the  “dissimilar"  pairs.  Here  we 
compute  the  correlation  between  the  particular  distance  measure 
and  the  "true"  distance  over  all  pairs  fq^  :  l£i£93,  -for  a  given 

subject.  By  analyzing  the  p^’s  and  q^'s  separately,  we  allow  -for 
the  possibility  that  a  particular  distance  measure  may  be  quite 
accurate  in  modeling  -fine  variations  in  similarity  (i.e.  small 
distances),  but  may  be  quite  inaccurate  in  modeling  -fine 
variation  among  pairs  that  are  each  composed  o-f  two  "dissimilar" 
words.  Furthermore,  in  practical  applications  one  may  only  need 
to  find  a  distance  measure  that  is  sensitive  to  the  degree  of 
similarity  in  pairs  of  "similar"  words  (e.g.  in  linguistic 
appr  o:.  1  mat  i  on  )  .  The  separate  analyses  also  give  a  distance 
measure  the  opportunity  to  be  linearly  related  to  "true" 
distance  with  two  (locally)  different  slopes  (see  Figure  3). 


»»**t*t**»*t**tt»t*t***f******* 

Insert  Figures  4  &  5  about  here 

For  each  distance  measure,  its  p  -correlations  for  the  15 

l 

subjects  are  summarized  by  a  line-plot.  The  19  line-plots  (one 
for  each  measure)  appear  in  Figure  4.  Analogous  line-plots  of 
the  q^ -correl ati ons  appear  in  Figure  5.  It  is  desirable  for  a 


measure  to  have  high  mean  and  median  correlation,  to  have  small 


dispersion  among  its  correlations  (i.e.  interquarti le  range)  , 


and 


to  be  -free  of  extremely  low  (i.e.  negative)  cor  rel  at  i  ons . 

Several  trends  are  clear  from  these  displays: 

(i)  There  is  a  great  deal  more  variability  between  the 
performances  of  the  various  distance  measures  on  "dissimilar" 
pairs  (Fig.  5)  than  on  "similar"  pairs  (Fig.  4):  the  means, 
medians,  and  i nter quart i 1 e  ranges  are  much  more  homogeneous  in 
Figure  4  than  in  Figure  5.  (Note  that  statistical  fluctuation 
would  actually  work  in  the  opposite  direction:  the  correlations 
(or  the  "dissimilar"  pairs  are  calculated  from  9  data  points, 
while  those  for  "similar"  pairs  are  calculated  from:  6  data 
points.)  This  immediately  suggests  that  more  caution  must  be 
exercised  when  selecting  a  distance  measure  to  distinguish 
between  varying  degrees  of  dissimi laritv. 

(ii)  On  the  "dissimilar"  pairs  (Fig.  5),  those  measures  which 

n 

perform  the  worst  (d_,  (d_)  ,d,,d  ,S_,S _,*))  are  measures  which 

Z  Z  I'm  Z  C- ' 

ignore  the  ordering  on  the  x-axis  (the  base  variable  axis). 
Conversely,  those  measures  which  perform  the  best  (q  ,q.,A  ,A.) 

CO  *  ©3  * 

are  measures  which  do  account  for  the  distances  on  the  x-axis  by 
looking  at  a-1 evel -sets.  This  distinction  is  quite  logical:  When 
measuring  the  distance  between  words  that  are  essentially 
"dissimilar"  (i.e.  have  nearly  disjoint  supports),  it  is  the  x- 
axis  that  carries  all  the  information  regarding  the  degree  of 
di ssi mi  1 ar i ty  between  the  membership  functions.  Distance 
measures  that  ignore  the  x— ordering  have  the  advantage  of  being 
unambiguously  defined  even  for  membership  functions  over  abstract 
(i.e.  unordered)  spaces,  but  such  measures  have  the  disadvantage 
of  being  insensitive  to  varying  degrees  of  dissimilarity  (e.g.  as 


in  pairs  ) .  In  the  "similar"  pairs  (Fig.  4),  the  membership 
■functions  within  a  pair  (p^  )  have  nearly  identical  supports. 

Hence  the  x-distance  is  not  critical,  and  we  find  both  types  o-f 
distance  measures  doing  well-  those  that  look  at  a-level-sets 
(notably  q  ,  4.  , &  >,  and  those  that  ignore  the  ordering  on  x 
(notabl y  S^) . 

(iii)  Among  those  measures  accounting  -for  x-ordering 

(q,.q  ,q  ,A  ,A  ,4  ,Q),  q  and  Q  are  especially  susceptible  to 
having  extremely  poor  correlation  with  "true"  similarity  ratings. 
This  occurs  -for  both  q.  -correlations  and  p  . -correl  at  i  ons.  Note 

l  l 

that  D  is  conceptually  di-f-ferent  -from  the  other  6  such  measures, 
possibly  accounting  -for  the  di-f-ference  in  performance. 

(iv)  Measure  is  arguably  the  worst  both  -for  "similar"  pairs 
and  for  "dissimilar"  pairs. 

(v)  Measures  Sj  and  are  clearly  the  best  in  terms  of  - 
correl ati ons,  among  those  measures  which  ignore  the  x— ordering. 
Their  superiority  in  the  "dissimilar"  setting  is  noteworthy 
because,  again,  x-distance  i_s  relevant  in  this  setting. 
Furthermore,  measure  performs  reasonably  well  (among  all 
measures)  in  the  "similar"  setting  also. 

(vi)  Quite  surpr i si ngl y ,  all  of  the  measures  with  consistently 
good  performance  <S^, q^, q^, 4^, )  share  the  following  property: 
they  concentrate  their  attention  on  a  single  value,  rather  than 
performing  some  sort  of  averaging  or  integration.  In  the  case  of 
S^,  attention  focuses  on  the  particular  x-value  where  the 
membership  function  of  AflB  is  largest;  in  and  4^  attention 
focuses  on  the  a-level-set  where  the  x-distance  is  largest;  in  q# 


and  A ^  attention  -focuses  on  the  x-distance  at  the  highest 
membership  grade.  Such  measures  are  generally  considered  unstable 
(and  hence  suspect)  in  many  mathematical  analyses.  Yet,  here  is 
strong  empirical  evidence  that  subjects  actually  behave  this  way: 
reduction  of  complicated  membership  functions  to  a  single  "slice1' 
may  be  the  intuitively  natural  way  for  human  beings  to  combine 
and  process  fuzzy  concepts. 

(vii)  The  consistently  good  performance  of  q^,  and  has 
significant  practical  implications.  These  measures  are  trivial 
to  compute,  relative  to  other  distance  measures,  and  they  have 
substantial  intuitive  appeal. 

(viii)  Distance  measure  R  was  proposed  as  a  refinement  of  V . , 
where  the  latter  is  used  in  the  first  stage  of  linguistic 
approx i mati on  and  the  former  is  used  in  the  second  stage 
(Bonissone  C21).  However,  the  empirical  results  show  na 
systematic  evidence  of  R  being  superior  in  the  "similar"  word 
setting  (Figure  4)  or  of  being  superior  in  the  "dissimilar" 
word  setting  (Figure  5). 

Recommendations. 

If  one  wants  to  select  a  distance  measure  that  performs  well 
in  the  long-run  on  a  broad  spectrum  of  subjects,  then  the 
aggregated  data  of  our  study  may  be  used  as  a  guide.  Measures 
S.,q4,A  ,  and  A .  consistently  distinguished  themselves  for  good 
performance.  If,  on  the  other  hand,  the  objective  is  to 
accurately  model  the  behavior  of  a  specific  individual  (e.g.  in 
the  linguistic  approximation  phase  of  an  expert  system  program), 
then  the  following  problem  must  be  acknowledged:  For  each 


L^4  ti 


distance  measure,  there  existed  some  subject  -for  whom  that 


distance  measure  performed  quite  poorly  (note  the  '‘minimum" 


values  on  Figures  4  and  5).  Therefore,  in  these  applications,  it 


would  be  ideal  to  determine  the  best  distance  measure  f or  the 


individual  of  interest.  This  could  be  accomplished  by  carrying 


out  an  experiment  analogous  to  ours,  but  on  the  specific 


individual,  and  in  the  relevant  context.  It  is  possible  that  the 


relative  performances  o-f  the  distance  measures  could  vary  -from 


one  context  to  another,  even  -for  a  -fixed  individual. 


In  many  cases,  the  -fuzzy  concepts  are  unambiguously  defined 


over  a  one-dimensional  space  (e.g.  in  our  study  of  probability 


words).  When  this  is  not  the  case,  then,  in  using  those  distance 


measures  that  do  account  for  the  ordering  on  the  base-var i abl e 


axes,  it  is  imperative  that  the  fuzzy  concepts  be  correctly 


located  in  a  space  of  the  appropriate  dimensionality. 
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Figure  I:  Direct  Estimation  Trial 
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